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ABSTRACT 

' ' The underlying i^ea of this wferk is that sedical 

students should learn to be effective problem solvers through their 
curricular experiences. The author studied 32. f o\irth year ■edxtral 
students from two colleges: 16 were from a college wxth a tradxtxonal 
curriculum; 16 were from a college whose curriculum included some 
_fiarl«±±^±s_at_pxobl£m^ioiien±e-d.Jt^^ 

assigned to one of four treatment conditions. Four students from each 
•college were provided with a set cf heuristics for clinxcal PJ:?l>lf ■ 
solvirig'and prompted to use them; four were provxded yxth heurxstxcs, 
but received no prompts for their use; four students from each 
college developed their own personal tests of problem solving 
h-uristics-.and were prompted to use them; four developed their 
personal test but received no prompting. .Subjects had initially 
worked through a paper case- from which' pretest measures were derived. 
Subjects who received the prepared set of heuristics were given aid 
in their use through another case. All subjects- worked through two 
additional cases from which posttest measures were derived. Tje 
results indicate that fourth year medical students perform better on 
clinical cases when they have been given powerful heuristics. 
^Author/EB) ' - , , 
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the ability to solve problems has for some time been proposed as a major 



goal of schooling. Some of the innovations of the Eight Year Study had their? 
source in the hope that schools could teach students t5 be more 'effective problem 
solvers 'cAikin, 1942; Smith, E.R. and Tyler, R.W., 1942). Some of the same hopes ^ 
and ideas are found in recent curricular innovations in medical schools (Ways, 
Loftus, a-nd Jones, 1973). The organization of much" of *^e medical school cur- 
. riculum around particular problems of patients is. relatively new. The underlying 
idea-that medical students should learn an effective problem solving strategy 
; through their curricular experiences.-goes back at least to Flexner's famous . - 

report (1910). , . • 

One of the' assumptions that has given problem solving a dentral place in the 
curriculum is that problem solving strategies and techniques learned with one 
• problem will be easily trans ferred/t'o other prob'lems. Thus, problem solving is 
■ seen as an activity that has significant generalizAble.aspects, This view of 

problem solving has recently been questioned by .researchers an'd curricular designers 
. lA medical education.. The questions have arisea because studies of clinical prob- ^ 
- " lem solving in simulations indicate that- performance is -case 'specific (Shulman, 
. 1974; McGuire, 1976)°, The correlation between performance on one case and per- 
foinnance on another case is low (less than .3, on average). Thus the assumption 
of transferability of problem solving skills has lacked support. 
' CoAtinuer findings -.of low correlations in performance from case to case 
^ 'weaken the argument of those who would make problem solving central in the medical 



curriculum. ^Such findings suggest that problem solving for a given case depends 
on knowledge or skill that is more-or-less unique to the condition encountered 

in..the-.case^.-J^--what .m.atters-is„the.-possession_of specifics- relevant to 
iiidiviaual cases j the curriculum should emphasize acquisition iOf as ma,ny and as 
varied a *se^ o£ specific's as possible. A curricular emphasis on the p^rocess of 
prakem solving makes sense orfly when there is evidence that'possession of some 
tactics or strategies of problem solving leads to improved performance. Re-- 
analysis of a recent stud^ of the role of heuristics in medical problem solving 
provides such evidence. 

The 'study - ' 

r. » 

Gordon (1973) studied 32.fo^h year medical students from two medical col- ^ 
leges: sixteen students from a coUege with a "traditional" curriculum; sixteen 
■students from a college whose curriculum included some early attempts at "problem^ 
oriented" teaching. Students in these groups were randomly assigned to one of 
four treatment conditions. Thus four students from each college were provided . 
vyith a set of heuristics for clinical problem solving' and were prompted to use the 
heuristics; four^were provided with heuristics but received no prompts for their 
use; four students from each college developed their own personal lists of problem 
solving heuristics and we.l prompted to use the heuristics; four developed their 
personal list of heuristics but received no prompts to use the heuristics. The 
•heuristics provided by Gordon are listed in Figure I. 

Subjects had initially worked through a "paper case*' from which pretest 
measures were derived. Those subjects who received the prepared set of heuristics 
were aided in using the heuristics as they worked- through another case. Then all 
subjects worked through two additional cases from which post test measures were 



derived. The four c^es, in order, are a. problem of exhaustion. wi%h a diagnosis 
of ulcerative colitis; a problem of fatigue and headache with a diagnosis of 
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Each piece of information requested by the problem 
solver should be related to a plan of attack for 
solving the problem. There should be a plan and a 
well defined purpose behind every question asked. 



2. 



3. 



No diagnostic hypothesis should be more specific or 
more general than the evidence on ^^f^^^Jj^^^^®^* 
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There should be at least two or three competihg 
hypotheses under- consideration at a particulaV time. 
Each piece of information should be evaluated with 
respect to all hypotheses presently under consideration. 



4. Whenever a new or revised hypothesis emerges, the 
information previously collected (particularly the 
information from the middle of the sequence of 
questions asked) should be reviewed. " Tfie problem 
solver should attempt to categorize the previously 
"elicited findings, as either tending to confirm or 
terming to discpnfirm his new hypothesis. 



5. 



When high cost (expensive, uncomfortable or risky) 
procedures are^eing considered to confirm a^ favored 
hypothesis, the problem solver should consider the 
possibility of lower cost procedures which migbt 
instead " rule out one or more diagnostic possibilities 
in order~to~make the- high cost procedure. unnecessary or 
to increase the probability tfiat the high cost pro- 
cedure will yield the definitive diagnosis.. 



FIGURE 



1. The heuristics provided to half of the subjects in the Gordon study. 
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hereditary- spherocytosis triggered- by mononucleosis; a problem of left chest 
pain with a diagnosis of mGltiple myeloma; a problem of nausea and vomiting' with 
a diagaosis of acute glomerulonephritis. • 

A number of measures are available bassd upon the record of diagnostic 
-hypotheses being entertained and actions taken as each subject worked through 
each paper case, 'p.e variables used in the analyses are^ "scoEe of Early Hypotheses. 
Number of Critical Findings Elicited, Cost of His toixy and 'Physical Examination, 
Cost^of Laboratory Work , and Accuracy of Definitive Diagnosis. Gordon (1973) 
provides a detailed description of these variables and the procedures used in 
their measurement. " * . 

Original analyses examined four d§gendent variables, one at a time. Scope, 
Critical. Findings, Overall Cost (the sum of Coasts of HfstoVy, Physical and 
Laboratory), and Accuracy were analyzed in a series, of ~2 (Schools) by 4 (Treatments). 
Anaiyses of Variance with the pretest scores on each variable used as a covariate. 
Only with Accuracy did the analysis suggest statistical significance .(F3. 23 = 2.69; 
p < .07), but because of the repeated tests a criterion'of p < .01 had beenchosen 
as the standard for statistical significance. . " ' 

A variety of ijicftcators suggested that a reanalysis might be fruitful. First 
the costs of laboratory in each case (and consequently the overall costs) are 
badly skewed, with mean roughly proportional to standard deviation within cells." 
Taking the log of this cost drives 'the distribution back towards a symmetric ■ , 
normal distribution and stabilizes^the variance. Second, the design 'of the.^study 
was 2 (School) by 2 (Heuristics) by 2 (Prompting), with real expectation of effects 
only for the Heuristics treatment, no expectation of effects associated with * 
-schools, and. the" effects of prompting uncertain, The dependent variables were . • 
•not independenc; the direction 'and , strength of their associations could be'taken 
into account in a multivariate analysis of variance. Finally, with so few degrees 



of freedom available it seemed best to use c.ovariates only if they were sig- 
nificant p'redictors of tlie dependent variables. ^ . ^ 

As preparation for the analysis the ^degree of association between the pre- 
test .variables and posttest variables was tested. It didn't approach statistical 
significance. Consequently the posttest variables were analyzed in an analysis 
of variance. This is done in two parts: first an analysis of the difference 
between scores on the two posttest cases to see whether an. interaction of" treat- 
ment with cases is present; second, an analysis of the sum of scores on^the two 
posttest cases to determine whether there is an overall effect for the design 
factors. The sijmmary statistics for these analyses are displayed in Table 1. 

^ The results in Table 1 suggest a possible sensitivity of treatment effects 
"tO: case- specific characteristics. ■ However, interactions are not statistically 
-signi-ficant. The analysis of posttest sums indica|es that school and school- 
related de'sign factors hav-e no apparent affect .on the^ outcomes of the study. T^e 
results also indicate that experience with and access to the prepared set of 
heuristics has an effect on medical problem solving that is likely to bg 
replicable. The effect can be described by the discriminant function separating 
the students, who worked with the prepared heuristics from the subjects who worked 
with-tljeiT personal heuristics. The standardized coefficients are indicated in 
Table 2. These results are what would be expected from knowledge of the 
heuristics. The first, second, fourth, and fifth ¥ieuristics urge a more system; 
- atic and cost-efficient search for and use of diagnostic information. This 
should lead to improved accuracy and lower costs and it does. The second and 
third heuristic urge the articulation of an appropriately broad set of diagnostic 
hypotheses. This ?hould lead to a wider range of early hypotheses, and it does. 
The set of heuristics does not bear in an obvious way .on the number of critical 
findings elicited. The relationship to the critical findings variable is the 
weakest in the set. 



.TABLE 1 

* ' 

••F" Values Associated With Edch of the Design Hypo'theses 
A. For Differences Between Scores on Post test Cases - 



Design Hypothesis 
SCHOOL 

HEURISTICS . h 
PROMPTING . 

# 

HxP 
.SxH . 
5xP 
SxHxP 



F Ratio ■ 

1-.42 
■ 2.36 »' 
2.42 

li-06 
.76 . 
.86 ' 

2.J29 



Probabilln;y 
Statement 

p < .26 ^ 
p <.^.08 
p ^ . 07 
. P < -41 



0 



, B. For Sums of Scores on Posttfest Cases; 

V * — ' 
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Design Hypothesis 



F Ratio 



Probability 
Statement 



SCHOOL 


» , - • .69 ' 


o / 




HEURISTICS , 


3.02 ■ % 




.03 


PROMPTING 


•1.27 ■ 


P < 


.32 


Hxf ■ 


1.14 

V 




■;37= 


^.SxH ' • " 


1.07 


p^ 


,41 


SxP 


.92 


✓ 




SxHxP . 


1.50 


■ p < 


.24 



*the number of degrees of freedom for each F is 5, 20 



, TABLE 2. 

Standardized Coefficients. For the-Discriminanf^unction 
Separating the Subjectsf Given Heuristics From the Others 

Variable " Coefficient 

Scope of Early Hypotheses ■ , '-^^ 

Cost of History and Physical . ^ -^'^S 

Cost of Laboratory Work -'^^ 

>imber- of Critical Findings Elicited -.29 

AiiCur'icy'of Definitive Diagnosis ' ' '^^ 
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ImpliGatjc nrs , » ■ • ^ . ' ^ ' 

Cldarly the critical experimental ef£ect--the effect of the set of prepared 
heuristics— is not so large as to be inescapably obvious in analysis. In order 
to find the effect you h^ve'to look for it. with care. The effett found, however, ' 
is precisely the eff ec-t expected and that should redouble our. confidence that it 
will be, robust aiid stable. * ''•*•"" 

The results- indicate that fourth- year medical, students perform better o^ 
clinical cases (more accurately and economically) when they have been given 
powerful heuristics. This effect appears de'spite the relatively brief introduction 
(experience in one case) that the experimental subjects had to the heuristics. It 
appears despite the fact that since the subjects w&re in their fourth year of 
medical school they were not clinical novices but had had time to develop adaptive 
•Strategies of-medical problem solving. And it appears despite the background of 
"noise" that -case-specific effects presents to the analysis. , ■ 
There can be effects- of strategy in. , clinical. problem solving. The effects 
■are seen most clearly when students are provided with more efficient proBlem 
solving, strategies. ■ Eff ective .problem solving heuristics appear to be teachable, 
The implication seems inescapable: .medical curricula should provide opportunities 
to learn powerful strategies for problem solving. • 

The Goi-don study is suggestive in other ways. Correlations between per- 
formance on one case and those on another case were low in this study as in 
others (see Table 5). In correlations alone we find no grounds for'supgort of 
attempts to teach problem solvijig in the curriculum. The Gordon study suggests 
that we will learn the value of intervening in the curriculum to teach problem 
solving only by carefully controlled study of such interventions. Curriculum 
Studies will have to become more experimental and interventionist, less passive 
and correlational. Correlational study can complement but cannot supplant the 
needed experimental studies. « " • , 
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TABLE, 3 



Average's "of Betvee^-Case Correlations- Among P.r6b],em. So I'virig^ Variables. ^ 



Scope 



Critical 
Findings 



History § 
Physical ' 
Costs 



Scope 


. .13 






Critical 
' Finding 


.09 


.32 
•t 




History. 5 
1 Physical 


.23 


.38' 


.52 


J Lab Costs 


-•.11 


.25 


.27 


i ' *> 
Accuracy* 


.-.10 


.10% 


. - lo 



Lab 'Costs 
(Log 

Transform^ 



.01 
.05 



Accuracy 



.00 



'correlations based' on deviations from celj means 
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The Gordon study ^also suggests two strategies that should prove Useful ^ 
moving us closer to an understanding of the process t>f problem^ salving in ■ 
important practical situations. ^' .6 first strategyjmight be called' disaggregat^n-- 
the development, of a comprehensive set of narrowly defined yaria]5les that can be' • 
aggregajed in various ways i This provides the analyst freedom to look for forins 
of aggregation that provide ^he' most stable, character iz^ioiT of problem salting 
behavior. .For example", Gordon monitored seven different elements ^h^t contri^fe 
to the cost of a work-up--the direct expense of 'a history, a physical examination,, 
laboratory tests^; the risks accompanying a 'physical examinaition, laboratory tests; , 
and the discomfort associated, with a physical ex^ajnination, laboratory ;tests^_4liese 
seven caji be aggregated' into the costs of a history, and '-physical and o'f laboratory 
tests as in the present analysis. Or they can be aggregated in other ways that 

.may reveal hidden, but stable aspects of clinical problem solving performance. 

"•■*(■ . ' . - ' - * 

The second strategy is an attempt to captfire thes constraints on the proble^ . 

solver and his intentions as accurately and comprehensively 'as possible. The 

Gordon study makes major steps towards a fuller recognition that the clinical ^ 

situation presents choices of " action that involve both costs and a probability 

of benefit. 'Currency, in both the medical curriculum and studies of .clinical 

problem solving^ the practitioner's problem is presented as one oiC reaching an / ^ , 

appr9priate conclusion-ri.fe- * a correct differential diagnosis.- This sub- 

• ' * * *■ 

stitution of a problem of knowing for a problem of acting Ts ^characteristic not ^ 

.only of the medical curriculum, but of curriculum generally, fhq instrumentation 

of the Gordon study suggests what i.iay be required, if we are to more>ccuTately 

represent the problem faced V the clinician (or .of the man of practice, m 

general). It is essential that we capture the full range of practical elements 

in the "space" of a problem if we are to develop more adequate ideas about what 

constitutes functional problem solving ability. " ^ 

11 
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